Natural Language Processing (NLP) engines enable computing systems to interact with and create content containing natural language (herein “language.”) For example, machine translation engines, artificial intelligence engines (e.g. digital personal assistants), spelling and grammar correction engines, parts of speech tagging engines, auto-summarization engines, image labeling engines, text input interfaces, etc. are all instances of NLP engines. One class of NLP engines can predict a next item (e.g. word or letter) in a sequence given previous items of the sequence. These NLP engines are referred to herein as sequence NLP engines.
In some implementations, sequence NLP engines can be implemented using machine learning engines. A “machine learning engine,” as used herein, refers to a component that is trained using training data to make predictions for new data elements, whether or not the new data elements were included in the training data. For example, training data can include a language corpus including sequences of words or letters. In some implementations, each sequence can correspond to a context, such as an image, a content item being summarized, a conversation with an AI agent, etc. A new data item can have parameters such as a beginning of a sequence and a similar context to that of the training data. Examples of machine learning engines include: neural networks, support vector machines, decision trees, Parzen windows, Bayes, clustering, reinforcement learning, and others. Machine learning engines can be configured for various situations, data types, sources, and output formats. These factors provide a nearly infinite variety of machine learning engine configurations.
The versatility of machine learning engines combined with the amount of data available can make it difficult to select types of data, sources of data, or training parameters and procedures that create sequence NLP engines that effectively predict the next items in a sequence.
The techniques introduced here may be better understood by referring to the following Detailed Description in conjunction with the accompanying drawings, in which like reference numerals indicate identical or functionally similar elements.